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AN  evaluation  OF  AFIT’S  graduate  programs  in 


OPERATIONS  RESEARCH  (GOR)  AND  STRATEGIC  AND  TACTICAL  SCIENCES  (GST) 

EXECUTIVE  SUMMARY 


At  the  request  of  the  head  of  the  Department  of  Operational  Sciences,  Air 
Force  Institute  of  Technology  (AFIT),  a  seven-member  committee  was  formed  to 
evaluate  two  graduate  programs  which  they  offer.  These  two  programs,  the 
Graduate  Operations  Research  (GOR)  and  Strategic  and  Tactical  Sciences  (GST) 
Programs,  result  in  a  Master  of  Science  Degree  in  Operations  Research.  After 
reviewing  these  programs  and  visiting  the  campus  at  Wright-Patterson  AFB,  near 
Dayton,  Ohio,  the  committee  has  concluded  that  both  programs  are  exceptionally 
effective  in  meeting  the  goal  of  preparing  military  officers  for  roles  requiring 
scientific  analysis  and  decision  making  skills.  In  general,  the  resident 
faculty  is  competent,  the  curricula  are  strong,  and  the  student  body  is 
energetic,  qualified  and  motivated.  Moreover,  most  graduates  of  these  programs 
are  employed  in  positions  that  require  the  skills  developed,  and  supervisors  are 
pleased  with  their  performance. 

The  predominantly  military  faculty  provides  some  unique  strengths  to  the 
program  as  well  as  a  few  drawbacks.  The  faculty  does  not  have  the  teaching 
experience  observed  at  civilian  universities,  but  they  make  up  for  this  with  a 
strong  dedication  to  teaching.  Technical  expertise,  due  in  large  part  to 
recently  completed  doctoral  studies,  is  exceptionally  high.  The  students 
identify  well  with  the  Air  Force  and  Army  faculty  and  are  highly  motivated  to 
study  the  subject  material,  especially  when  the  faculty  is  able  to  demonstrate 
its  relevance.  The  faculty  is  very  responsive  to  student  needs,  a  trait  which 
is  often  not  found  at  civilian  institutions. 


The  required  core  curriculum  is  uncommonly  strong  compared  to  civilian 
programs.  Students  are  provided  "brush-up"  review  courses  to  give  them 
prerequisite  skills.  Application  of  computers  (of  all  sizes)  is  growing 
rapidly.  Exceptional  financial  and  facility  support  is  evident  in  the  rapid 
growth  of  available  computer  hardware  and  software. 

Although  the  committee  found  both  graduate  programs  strong  and  effective, 
we  noted  several  areas  of  concern.  The  most  notable  problems  involve  faculty 
manning  and  institutional  support  from  Air  Force  agencies.  Specifically,  the 
teaching  experience  of  the  military  faculty  is  quite  low,  and  many  have  not 
completed  their  doctoral  dissertation  requirements.  The  head  of  the 
Department  of  Operational  Sciences  will  be  leaving  with  no  designated 
replacement;  the  Department  of  Mathematics  and  Computer  Science  had  been 
without  a  department  head  for  over  two  years.  (A  new  head  was  hired  on  6  July 
1987.)  In  addition,  both  departments  are  undermanned.  The  support  the  faculty 
and  students  receive  from  Air  Force  personnel  agencies  is  somewhat 
debilitating  to  the  effectiveness  of  the  programs.  These  problems  are 
certainly  not  of  crisis  proportion  and  may  be  cyclical  in  nature  due  in  large 
part  to  the  predominantly  military  faculty.  However,  the  Department  Head 
spends  considerable  time  procuring  future  faculty  members  and  the  students 
must  invest  significant  time  and  effort  to  assure  that  their  follow-on 
assignment  is  consistent  with  their  skills  and  desires. 

Ue  have  two  recommendations  to  ameliorate  these  personnel  problems: 
extended  tours  for  military  faculty  and  an  AFIT  Operational  Sciences  (AFIT/ENS) 
"advocate"  at  the  Air  Staff  level.  Tours  could  be  extended  by  adding  two 
years  to  initial  assignments  and  by  establishing  bona  fide  tenured  positions 
for  military  faculty.  Creating  a  Senior  268X  Advisory  Board,  consisting  of 


Coraraanders/Directors  of  major  user  agencies,  would  place  more  attention  on 
careful  matching  of  graduate  qualifications  and  job  requirements.  (The  Army 
has  a  comparable  FA49  Advisory  Board,  which  meets  twice  a  year.)  High-level 
support  for  AFIT/ENS  and  its  programs  will  reap  other  benefits  as  well, 
including  a  significantly  reduced  administrative  burden  on  the  faculty  and 
increased  status  of  the  programs  in  the  eyes  of  prospective  students. 

Another  area  cited  for  potential  improvement  is  faculty  development,  which 
would  benefit  from  a  comprehensive  and  ongoing  faculty  development  plan.  As 
part  of  this  plan,  all  new  instructors  should  receive  instruction  on 
techniques  appropriate  for  graduate-level  teaching  before  they  begin  their 
first  quarter.  An  "instructional  mentor"  program  might  also  be  effective. 

A  few  minor  areas  of  criticism  involve  specific  curriculum  issues. 

Students  would  benefit  from  an  introductory  block  of  instruction  about 
operations  research  as  a  profession,  including  its  history  and  development. 
Several  specific  subjects  are  covered  too  lightly.  The  integration  of 
economics  in  the  programs  is  weak.  Solutions  to  these  problems  would  be 
fairly  easy  to  implement. 

We  have  discussed  all  items  in  detail  in  the  body  of  this  report.  Again, 
the  committee  unanimously  feels  that,  on  the  whole,  the  programe  offered  by 
the  Department  of  Operational  Sciences  are  viable,  creditable,  and  very 
effective  in  meeting  their  stated  goals. 


BACKGROUND 


Col  Michael  O'Connell,  the  Head  of  the  Department  of  Operational  Sciences 
In  the  School  of  Engineering,  Air  Force  Institute  of  Technology  (AFIT) 
requested  that  a  committee  be  formed  to  evaluate  two  programs  offered  by  his 
department.  The  US  Air  Force  Academy  accepted  the  task  and  organized  the 
following  team: 


Lt  Col  Peter  Knepell,  Chairman 
PhD  In  Operations  Research,  Cornell  University 
Deputy  Head,  Department  of  Mathematical  Sciences 
US  Air  Force  Academy,  CO 

Prof  William  Lesso 

PhD  In  Operations  Research,  Case  Institute  of  Technology 

Professor  of  Operations  Research,  Department  of  Mechanical  Engineering 

University  of  Texas,  Austin,  TX 

Rear  Adm  (Ret)  Kleber  Masterson,  Jr 

PhD  In  Physics,  University  of  California,  San  Diego 

Chairman,  Education  Committee,  Military  Operations  Research  Society 

Vice  President,  Booz,  Allen  &  Hamilton  Inc.,  Alexandria,  VA 

Col  Kenneth  Fleming 

PhD  In  Economics,  University  of  California,  San  Diego 
Vice  Dean  and  Professor  of  Economics 
US  Air  Force  Academy,  CO 

Lt  Col  (USAFR)  R.  Warren  Langley 

PhD  in  Operations  Research,  Georgia  Institute  of  Technology 
Director,  Research  &  Development,  United  Airlines,  Chicago,  IL 

Lt  Col  Michael  Anselmi 

PhD  In  Quantitative  Economics,  University  of  Colorado 
Acting  Head,  Department  of  Economics  and  Geography 
US  Air  Force  Academy,  CO 

Maj  Vernon  Francis 

PhD  in  Management  Science,  University  of  California,  Los  Angeles 
Tenure  Associate  Professor,  Department  of  Management 
US  Air  Force  Academy,  CO 


This  evaluation  is  based  on  a  review  of  the  programs  at  the  Department  of 


Operational  Sciences,  School  of  Engineering,  AFIT  (AFIT/ENS),  research  on 
comparable  programs,  interviews  with  AFIT  graduates  and  their  supervisors,  and 
campus  visits  by  members  of  the  committee.  We  researched  procedures  typically 
followed  by  accreditation  boards  and  decided  to  follow  the  procedures  and 
criteria  established  by  the  Accreditation  Board  for  Engineering  and  Technology 
(ABET),  as  expressed  in  their  1985  Annual  Report.  This  body  has  accredited 
operations  research  programs  including  the  one  offered  by  the  Department  of 
Mechanical  Engineering  at  the  University  of  Texas.  Before  visiting  the  AFIT 
campus  at  Wright-Patterson  AFB,  members  of  the  committee  reviewed  materials 
provided  by  the  Department  of  Operational  Sciences  and  interviewed  several 
graduates  of  their  programs  who  are  on  the  faculty  at  the  Air  Force  Academy. 

On  two  separate  visits  we  Interviewed  faculty  and  students,  observed  classes 
and  thesis  defenses,  toured  the  facilities,  and  examined  course  materials  in 
greater  depth. 

The  body  of  this  report  follows  the  general  criteria  suggested  by  the  ABET 
Report.  Each  area  evaluated  is  discussed  separately,  including  observations 
of  members  of  the  committee  and  their  recommendations. 

Beforehand,  however,  we  must  commend  the  spirit  of  cooperation  we 
received  from  Col  O’Connell,  his  faculty,  and  other  members  of  the  staff  and 
faculty  of  the  School  of  Engineering.  We  also  want  to  thank  Lt  Col  Joseph 
Falx  for  his  superb  job  in  coordinating  our  schedules  and  making  our  visits  as 
productive  as  possible  — this  was  a  big  task  and  it  couldn't  have  been  done 
better. 
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CRITERIA 


The  criteria  used  are  Intended  to  provide  a  basis  for  a  thorough 
evaluation  of  the  program.  It  is  difficult,  indeed  inappropriate,  to 
establish  firm  guidelines  on  how  to  measure  a  program's  effectiveness  in 
satisfying  a  set  of  criteria.  The  ABET  Report  is  quick  to  point  out  that  the 
criteria  should  be  applied  with  sufficient  flexibility  to  allow  for  an 
institution's  particular  strengths,  characteristics,  and  ideals.  In  fact,  the 
report  aptly  states:  "[The  criteria]  are  intended  to  encourage  and  stimulate 
and  not  to  restrain  creative  and  imaginative  programs."  This  is  the  spirit  of 
our  comments  to  follow. 

A.  Faculty 

1.  Observations .  The  size  and  quality  of  the  faculty  is  the  heart  and 
soul  of  any  program.  The  Department  of  Operational  Sciences  has  an  energetic, 
vital  faculty  consisting  of  nine  Air  Force  officers,  one  Army  officer,  and  one 
civilian.  The  authorized  strength  is  13  instructors.  This  summer,  two  new 
civilian  professors  will  be  added,  and  the  Department  Head,  Col  O'Connell, 
will  retire.  Of  the  11  assigned  faculty  members,  5  have  not  finished  their 
doctoral  dissertation  requirements.  Among  the  military  faculty,  the  average 
graduate-level  teaching  experience  is  1.9  years.  The  civilian  Instructor  has 
17  years  of  experience.  Faculty  members  attended  a  wide  range  of  respected 
graduate  schools.  This  is  also  true  of  officers  "in  the  pipeline"  for  PhDs 
and  future  assignment  to  AFIT.  Typically,  a  new  military  member  attends  the 
Air  Force's  Academic  Instructor  School  at  Maxwell  AFB  during  the  first 
academic  year.  This  three-week  course  is  specifically  developed  for  the  AFIT 
faculty.  Although  the  department  intends  to  provide  each  faculty  member  a 
free  quarter  every  year  for  professional  development  and  research,  low  manning 


interferes  with  this  goal.  The  Department  of  Mathematics  and  Computer  Science 
instructs  four  courses  in  the  required  core  curriculum  and  offers  several  others 
supporting  minor  specialties,  such  as  the  reliability  and  maintainability 
sequence.  The  department  has  been  directed  by  a  succession  of  interim  and  acting 
heads  over  the  past  two  years.  A  full-time  civilian  head  was  appointed  in  July 
1987.  Currently,  the  department  has  17  instructors  and  the  authorized  strength  is 
21.  Two  of  the  assigned  faculty  have  not  completed  their  doctoral  requirements. 
On  the  whole,  the  department  plays  a  support  role  by  offering  and  teaching 
courses  in  mathematics,  probability,  statistics,  and  computer  science.  Only  one 
instructor  is  heavily  involved  with  the  students  in  the  operations  research 
programs  and  he  is  a  thesis  advisor  for  several  students. 

2.  Strengths .  The  faculty  are  a  dedicated  and  motivated  group,  committed  to 
their  students  and  AFIT's  mission.  They  take  a  personal  interest  in  their 
students  and  do  everything  possible  to  ensure  successful  completion  of  the 
program.  The  military  members  are  usually  fresh  from  graduate  studies;  thus, 
while  their  teaching  experience  is  low  compared  to  their  civilian  counterparts, 
their  knowledge  base  is  current  and  often  relevent  for  military  applications. 

The  faculty's  military  experience  in  operations  research  lends  important 
credibility  to  their  teaching.  The  faculty  is,  and  should  remain,  head  and 
shoulders  above  their  civilian  counterparts  in  this  area.  Furthermore,  the 
faculty  research  program  seems  excellent  and  relevant  to  current  Air  Force  work 
in  the  field.  An  energetic  faculty  encourages  students  to  research 
state-of-the-art  developments,  such  as  multiple  criteria  decision  making  and 
artifical  intelligence.  Through  end-of-course  critiques  and  interviews  with 
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students,  the  committee  has  observed  that  the  students  are,  with  very  few 
exceptions,  very  happy  with  their  Instructors,  especially  the  military  ones. 
Student-faculty  Interaction  Is  exceptional.  Students  receive  thorough  and 
supportive  counseling  and  advice.  The  faculty  seems  very  willing  to  react  to 
student  criticisms  and  suggestions. 

3.  Weaknesses.  The  most  obvious  weakness  observed  Is  the  large  number  of 
Instructors  who  have  not  yet  completed  their  doctoral  dissertations,  reducing 
the  credibility  of  the  Institution,  and  thus  Its  graduates,  In  the  academic 
world.  This  problem  Is  exacerbated  by  the  heavy  workloads  Imposed  on  the 
faculty,  especially  with  respect  to  thesis  advising.  Most  Instructors  have 
plans  to  complete  their  dissertations  within  a  year  or  two,  but  this  Is  a  burden 
which  distracts  them  from  their  primary  teaching  duties.  In  addition,  the 
department  head  and  one  of  his  senior  faculty  spend  an  Inordinate  amount  of  time 
recruiting  future  faculty  members.  This  process  usually  Involves  sponsorship 
for  PhD  studies  enroute  to  a  teaching  assignment.  AFIT  must  continually 
coordinate  this  recruiting  with  several  Air  Force  agencies. 

While  the  level  of  practical  Air  Force  experience  and  Involvement  In 
military  research  Is  high,  production  of  publishable  research  papers  and 
Involvement  In  professional  societies  Is  relatively  low.  Even  though  the  low 
teaching  experience  of  the  faculty  Is  mitigated  by  the  strengths  noted  earlier, 
there  are  some  distinct  disadvantages.  Academic  rank  is  comparatively  low  and 
could  detract  from  the  program's  prestige.  We  had  some  feedback  that  new 
instructors  occasionally  have  trouble  adapting  to  their  new  role;  their  lesson 
plans  are  weak  and  their  lectures  are  ineffective  In  clarifying  the  material. 


4.  Recommendations .  Several  solutions  are  available  to  solve  the  problems 
with  faculty  experience  and  PhD  completion.  A  tenure  system  could  be 
established  for  military  faculty  with,  perhaps,  four  year  renewable  options  much 
like  the  system  at  the  Air  Force  Academy.  (AFR  36-20  and  AUR  36-1  allow  a 
similar  kind  of  tenure.)  This  option  Is  not  without  potential  problems.  Tenured 
Instructors  are  removed  from  their  military  field  of  specialty,  possibly  leading 
to  promotion  pass-over.  Conversely,  officers  could  gain  too  much  military  rank 
for  their  position.  Extending  military  assignments  to  six  years  should  also  be 
considered.  Hiring  more  civilian  faculty  members  could  provide  stability  and 
expertise  without  some  of  the  potential  problems  of  military  Instructors; 
however,  this  could  lead  to  a  lack  of  flexibility  and  responsiveness  In  the 
department  (one  of  the  present  strengths)  and  a  consequent  reduction  of  student/ 
Air  Force  support  and  Interest.  The  answer  should  be  a  judicious  mix  of 

civilian  and  military  faculty,  with  a  clear  emphasis  on  the  military.  As  a 

point  of  Information,  the  Naval  Postgraduate  School  has  over  a  90X  civilian 
faculty,  many  with  20-25  years  of  tenured  service.  Defining  the  best  military/ 
civilian  mix  Is  a  difficult  problem,  but  one  which  can  be  addressed  with 
prestated  goals,  e.g.,  three  tenure  military,  plus  three  civilian  and  seven 
military  faculty  serving  tours  of  four  to  six  years.  The  committee  feels  that 

the  military  presence  is  paramount  due  to  the  nature  of  the  programs,  but  It 

should  be  supplemented  with  a  stable  blend  of  compatible  civilian  experience. 
With  the  two  civilian  members  that  will  be  hired  this  summer,  the  department  Is 
definitely  headed  toward  greater  stability. 

The  relative  weaknesses  in  faculty  research  and  publishing  should 
be  addressed  by  encouraging  more  participation  In  professional  meetings  and 
Joint  student-faculty  papers  based  on  thesis  work.  However,  top  flight 


researchers  at  civilian  schools  often  cannot  relate  to  students'  problems  and 
concerns;  the  AF1T  faculty  appears  to  be  very  good  at  the  latter,  which  is  far 
more  important. 

Concerning  the  teaching  experience  issue,  we  do  not  feel  that  attending  the 
Academic  Instructor  School  is  sufficient  for  new  graduate-level  instructors. 
Graduate-level  teaching  at  AFIT  may  require  more  detailed  preparation  in  the 
AFIT  environment.  An  internally  managed  faculty  development  program,  aimed  at 
improving  teaching  effectiveness,  might  be  more  productive.  A  short  introductory 
course  during  the  summer  and  a  new  instructor/mentor  program  Involving  practice 
lessons  and  tips  on  lesson  planning  and  curriculum  development  could  be  the 
first  phases  of  a  comprehensive  program.  The  Department  Head  should  also  ensure 
that  new  instructors  are  monitored  and  counseled  early  in  their  tours. 

Because  the  faculty  is  very  open  to  student  criticism  and  comments,  we 
caution  against  making  changes  based  simply  on  student  perceptions.  Many  times 
there  are  sound  pedagogical  reasons  to  make  the  students  "suffer.” 

Recruiting  new  faculty  members  is  also  a  major  problem.  The  burden  on  the 
department  would  be  greatly  relieved  if  it  could  receive  high-level  support  and 
assistance.  We  recommend  a  "force  manager"  for  the  268X  and  related  career 
fields  be  created  at  the  Air  Staff  level  to  handle  this  and  other  issues.  We've 
enumerated  other  benefits  of  having  such  an  advocate  later  in  this  report. 


B.  Curricular  Objective  and  Content 

1.  Observations.  The  curriculum  is  a  strong,  structured  operations  research 

program  which  leads  to  an  excellent  degree  in  this  field.  The  core  curriculum 
stresses  courses  in  mathematical  sciences  (including  probability  and  statistics), 
mathematical  programming,  stochastic  processes,  and  applications.  GOR  students 
enter  in  the  early  summer  and  GST3  enter  in  September.  The  school  takes 
advantage  of  this  by  offering  a  short  term  of  review  courses.  The  GOR  and  GST 
programs  have  different  sets  of  courses  in  this  short  term.  This  was  explained 
as  necessary  due  to  the  difference  in  backgrounds  and  abilities  of  students,  as 
well  as  differences  in  the  curricula.  Students  in  the  GOR  program  are  asked  to 
select  an  emphasis  area  and  those  in  the  GST  program  must  take  a  specialization 
sequence.  Every  student  must  write  a  thesis.  The  theses  we  observed  or  read 

covered  a  range  of  topics,  but  most  were  applications  of  simulation. 

2.  Strengths.  The  core  program  including  the  thesis  requirement  is  the  heart 

of  the  curriculum  and  its  strongest  feature.  Clear  structure  provides  focus  and 

direction  to  the  students.  On  the  whole,  the  level  and  pace  appear  to  be  good. 
Starting  the  program  with  a  short  term  of  review  courses  is  an  outstanding  idea. 
Students  ultimately  achieve  a  balance  of  theory,  design  and  practice  necessary 
for  a  good  graduate-level  program  in  a  technical  field.  The  thesis  requirement 
is  an  excellent  way  to  achieve  several  goals  that  ABET  feels  are  important, 
including  demonstration  of  competency  in  problem  analysis  and  structure,  research 
skills,  and  written  communication.  Students  of  the  program  are  motivated  by  the 
applications  orientation,  and  graduates  feel  confident  that  they  have  the  ability 
to  apply  their  skills.  The  application  of  computers,  both  micro-  and  mainframe. 


In  course  work  is  growing  rapidly.  Individual  members  of  the  committee  have 
singled  out  several  courses  and  areas  that  appear  particularly  strong: 
probability,  statistics,  reliability,  maintainability,  linear  programming,  and 
artificial  intelligence.  Many  of  the  courses  make  excellent  use  of  relevant 
problems  in  military  operations  research.  Maintaining  course  books  in  the 
departments  is  an  effective  way  to  insure  continuity  between  course  offerings. 

While  the  pace  could  be  criticized  as  being  too  Intense,  most  of  the 
students  and  faculty  see  it  as  a  necessary  evil  given  the  need  to  cover  so  much 
material.  Course  content  and  sequencing  has  to  be  constantly  reviewed  to  insure 
consistency  and  balance.  We  are  reasonably  convinced  that  this  is  being  done. 
For  example,  a  concern  about  the  lack  of  time  to  cover  stochastic  processes 
seems  already  to  be  "on  the  mend”;  the  department  is  planning  to  increase  this 
course  from  3  to  A  credit  hours,  thus  increasing  classroom  contact  hours  by 
33%. 

3.  Weaknesses. 

a.  Introduction  to  Operations  Research.  Many  of  the  students  selected 
to  these  programs  by  the  Air  Force  do  not  understand  clearly  what  operations 
research  is.  Moreover,  they  have  little  appreciation  for  the  history  of 
operations  reseach,  its  development,  or  its  close  association  with  planning, 
analysis,  and  evaluation  of  military  operations.  Thus,  students  do  not  gain  an 
appreciation  for  operations  research  until  they  are  well  into  their  program. 

The  solution  recommended  below  should  easily  correct  this  deficiency. 


b.  Integration  of  Economics  into  the  Program.  To  quote  one  member  of 
the  committee:  "This  is  the  one  area  that  is  weakest  in  the  current  program." 
The  GOR  students  must  take  two  courses  in  Economic  Analysis  (OPER  531  and  631); 
these  are  electives  for  the  GST  students.  The  students  are  dissatisfied  with 
this  part  of  their  curriculum;  they  don't  see  the  relevance  and  are  not 
Interested  in  the  subject,  largely  because  most  of  the  problems  and  examples 
used  are  not  related  to  military  operations  research.  Some  fundamental 
principles  of  economics  (e.g.,  present/f uture  value,  costing,  cost-benefit 
analysis,  regression  analysis,  etc.)  are  scattered  throughout  the  core. 
Unfortunately  students  do  not  see  the  subject  for  what  it  is:  a  field  that 
spawned  many  operations  research  techniques  and  one  which  provided  a  foundation 
for  much  of  operations  research. 

c.  GOR  versus  GST  Programs.  The  objectives  of  the  GOR  and  GST  programs 
are  different,  so  their  curricula  vary  somewhat.  Despite  these  differences, 
however,  both  groups  receive  a  Master  of  Science  Degree  In  Operations  Research. 
It  appears  that  the  GST  program,  is  a  "watered  down”  version  of  the  GOR  Program, 
especially  in  the  mathematics  area.  This  may  give  GST  students  the  feeling  that 
they  are  "second-class"  citizens.  The  GST  program  includes  much  nuclear  weapons 
physics  (e.g.  theory  of  how  energy  is  released)  that  appears  excessive  and  might 
be  traded  for  more  rigorous  mathematics.  Regardless  of  whether  the  problems 
incurred  by  two  different  programs  are  real  or  perceived,  they  need  to  be 
addressed. 

d.  Probability  and  Statistics.  GST  students  must  take  only  two 
quarters  of  probability  and  statistics.  GORs  take  two  quarters  of  probability 
and  statistics  and  then  an  applied  statistics  elective.  These  difficult 
concepts  are  fundamental  to  the  OR  profession  and  yet  are  often  misapplied  by 
practitioners.  It  takes  most  students  three  passes  through  the  concepts  for 


them  to  take  hold 


e.  Interdisciplinary  Team  Management.  Operations  research  as  an 
interdisciplinary  activity  is  beginning  to  make  a  comeback.  No  current 
university  program  addresses  the  skills  necessary  to  organize  and  manage  such 
an  interdisciplinary  team. 

£.  Thesis  Research.  While  the  committee  feels  that  the  thesis  is  an 
indispensable  and  pedagoglcally  important  requirement,  several  negative 
observations  are  worth  mentioning.  One  committee  member  felt  that  the  theses 
were  closer  to  "Master's  Projects."  It  appears  that  nearly  all  students  select 
their  thesis  topics  from  a  provided  list,  and  most  of  these  are  applications  of 
simulation.  The  result  is  less  effort  in,  and  hence  less  benefit  from,  the 
problem  identification  and  definition  phases  of  the  thesis  research.  We 
recognize  that  there  is  a  tradeoff  between  the  benefits  gained  through  student 
definition  of  thesis  topics  and  those  gained  through  doing  research  directly 
applicable  for  a  DOD  organization.  Despite  our  observations  or  impressions,  we 
are  convinced  that  the  thesis  program  is  very  effective  in  accomplishing  its 
goals,  as  evidenced  by  comments  from  graduates  (see  paragraphs  2  and  3,  page 
20). 

g.  Miscellaneous  Concerns.  The  rapid  Increase  in  microcomputer 
availability  may  hurt  some  students  who  are  not  as  "computer  literate"  as 
others.  Concerning  other  specific  courses,  one  member  of  the  committee  feels 
that  the  linear  programming  course  should  include  a  discussion  of  Karmarkar's 
algorithm  and  application  of  standard  software  packages.  Another  member  would 
like  to  see  a  course  or  set  of  courses  in  production/project  management  —  a 
sequence  that  would  be  applicable  to  military  maintenance  and  overhaul  facilities. 


Game  theory,  data  analysis,  negotiations  analysis,  and  physics  also  appear  to  be 
lightly  treated  or  not  covered.  Several  courses  were  relatively  weak  in 
problems  and  examples  from  military  operations  research.  Concern  was  expressed 
regarding  the  significant  number  of  "remedial"  and/or  "survey"  courses  compared 
to  other  quality  graduate  schools,  which  assume  prerequisites  are  satisfied 
before  entering  the  program  or  through  no-credit  courses. 

4 .  Recommendations 

a.  Introduction  to  Operations  Research.  The  committee  feels  very 
strongly  that  students  in  the  two  programs  should  be  provided  with  an 
appreciation  for  operations  research  as  a  subject  and  career  field  early  in 
their  program.  Under  present  structure,  students  get  a  brief  introduction  in 
the  second  quarter  with  the  Deterministic  Methods  course  (OPER  663)  and  a  final 
treatment  in  Analysis  for  Defense  Systems  (OPER  742)  or  Systems  Analysis  and 
Defense  Planning  (TACT  670),  which  occur  late  in  the  course  of  study.  We 
recommend  that  the  students  receive  a  "warm-up"  familiarization  block  of 
instruction  either  in  the  short  summer  terra  or  in  the  first  full  quarter.  This 
block  of  instruction  should  include  the  history  of  military  operations  research, 
introduction  to  the  scientific  method  and  the  systems  approach,  and  a  discussion 
and  illustration  of  some  of  the  techniques  that  are  now  covered  in  OPER  663. 

Some  discussion  of  the  interdisciplinary  and  team  nature  of  early  operations 
research  should  also  prove  beneficial.  This  would  have  the  advantage  of 
motivating  students  for  the  subject  and  course  of  study  to  follow.  As  a  side 
benefit,  the  OPER  663  course  could  then  drop  some  material  and  cover  the 
remaining  subjects  in  greater  depth. 
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b.  Integration  of  Economics  into  the  Programs.  The  students  should 
discover  the  worth  of  economic  theory  and  its  relationship  to  widely  used 
operations  research  techniques.  Several  texts  are  available  for  this  type  of 
integration,  the  most  notable  being  Economic  Theory  and  Operations  Analysis,  by 
William  J.  Baumol,  and  the  classic  Hitch  and  McKean  book  on  defense  decision 
making.  The  text  could  easily  form  the  basis  for  a  two-quarter,  graduate-level 
course  sequence  that  would  be  extremely  helpful  to  an  operations  researcher  who 
wanted  to  learn  economics.  In  addition,  many  of  the  topics  covered  in  the 
Baumol  text  are  complementary  to  those  covered  in  other  courses  in  the 
curriculum  (e.g.,  game  theory,  duality  in  linear  programming,  marginal  analysis, 
etc.).  Thus,  the  faculty  could  remove  certain  topics  from  other  courses,  leaving 
them  for  the  economics  «equence.  Further,  problems  and  examples  with  military 
relevance  should  replace  the  almost  purely  non-military  ones.  A  review  of 
courses  taught  by  the  Engineering-Economic  Systems  Department  at  Stanford  might 
prove  beneficial.  Several  members  feel  very  strongly  that  econometrics  be  a 
required  third  course  in  this  sequence.  Other  committee  members  feel,  however, 
that  time  pressures  dictate  that  it  should  be  an  elective.  These 
recommendations  would  serve  the  dual  purposes  of  showing  the  students  the 
complementary  nature  of  the  two  disciplines,  while  elucidating  the  economic 
aspects  of  operations  research  problems.  Finally,  the  GST  students  should  be 
encouraged  or  required  to  take  a  minimum  of  one  course  in  economic  analysis. 

c.  GOR  versus  GST  Programs.  While  the  need  for  two  separate  programs 
was  justified  to  us  very  well  by  the  faculty,  the  students  rarely  get  to  choose 
their  program.  From  the  beginning,  they  pursue  separate  tracks  toward  the  same 
degree.  This  is  especially  true  in  the  first  required  mathematics  course — 


MATH  503  (GST)  versus  MATH  509  (GOR).  This  early  tracking  leads  to  some 
undesirable  perceptions.  We  recommend  the  department  consider  having  both 
programs  start  with  a  common  core  in  the  short  review  terra  and  first  quarter. 

d.  Three  quarters  of  probability  and  statistics,  or  deliberate  rein¬ 
forcement  of  key  concepts  in  other  courses,  is  strongly  recommended. 

e.  Operations  research  as  an  interdisciplinary  activity  is  worthy  of 
research  and  coverage  in  the  curriculum.  One  way  to  begin  consideration  of  this 
topic  might  be  through  guest  lectures  on  subjects  relevant  to  military  operations 
research  by  historians,  physical  scientists,  psychologists,  sociologists,  project 
managers,  and  others.  The  faculty  might  also  research  and  teach  techniques  for 
integrating  team  efforts  and  the  concepts  of  project  management. 

f.  Miscellaneous.  Under  the  present  system,  students  do  not  often 
transfer  credits  or  validate  courses  from  other  programs  or  schools.  In  addition, 
they  typically  must  take  all  review  courses  offered  during  the  short  review  terms. 
We  recommend  some  validation/transfer  credit  to  allow  brighter  students  more 
flexibility  with  their  programs.  Current  courses  could  also  include  materials  on 
game  theory,  data  analysis,  negotiations  analysis,  and  goal  programming.  Some  of 
the  linear  programming  topical  coverage  might  be  reduced  to  compensate  for  this. 

A  course  on  "modern  physics"  should  be  included,  because  an  OR  practitioner  needs 
to  understand  modern  technology.  And,  as  for  the  economics  courses,  we  recommend 
continued  efforts  to  find  and  include  problems  and  examples  from  military 
operations  research.  In  a  different  area,  we  recommend  that  the  school  look  at 
the  "computer  literacy"  issue.  The  Department  may  want  to  provide  a  short  course 
on  microcomputers  for  the  entering  students  to  familiarize  them  with  the  operating 
systems,  word-processing,  spread  sheets,  graphics,  etc.,  that  will  be  available. 
APIT/ENS  needs  to  develop  a  comprehensive  plan  for  the  thoughtful  and  purposeful 
Integration  of  microcomputing  across  the  entire  curriculum. 


C.  Student  Body 

In  general,  the  committee  found  this  area  to  be  outstanding.  We  have  only  a 
few  recommendations  to  make  after  the  discussing  our  observations. 

1.  Admissions  Policy  and  Student  Performance.  Students  apply  for  the 
program  by  submitting  an  application  for  AFIT  selection.  Marginal  applications 
are  reviewed  by  the  School  of  Engineering  and  Department  of  Operational 
Sciences.  The  GOR  program  has  the  only  published  minimum  undergraduate  GPA, 
which  is  3.00.  While  the  School  of  Engineering  would  prefer  a  3.00  or  better 
GPA,  the  department  may  waive  this  preference  after  reviewing  other  factors, 
including  GRE  scores.  The  AFIT  Central  Selection  Board  and  the  Army’s 
equivalent  then  approve  18-24  students  for  entry  into  each  program.  They 
typically  identify  the  officers  with  weapon  system  experience  for  the  GST 
program.  The  resulting  student  body  consists  of  a  heterogeneous  mix  of  Air 
Force,  Army,  and  foreign  service  officers.  The  committee  feels  that  this 
diversity  is  good  for  the  programs.  Data  for  the  current  class  and  the  three 
previous  classes  are  in  Table  1. 


CLASS 

NR 

Undergraduate  GPA 
Aver  |  Low/High 

GRE  Average 
Verb  |  Quant 

AFIT  GPA 

Aver  |  Low/High 

87D 

24 

3.20 

2.20/3.83 

515 

682 

— 

- /  — 

86D 

20 

3.27 

2.74/3.93 

526 

696 

3.62 

3.02/4.00 

GOR 

85D 

22 

3.19 

2.61/3.81 

532 

686 

3.49 

2.90/3.94 

84D 

18 

3.01 

2.11/3.67 

503 

700 

3.56 

mm 

19 

3.12 

2.28/3.87 

536 

684 

— 

— /— 

GST 

23 

2.50/3.63 

557 

714 

3.54 

3.16/3.99 

■Iv 

22 

3.12 

2.23/3.89 

605 

673 

3.55 

3.27/4.00 

84M 

22 

3.05 

2.44/3.80 

555 

694 

3.68 

3.23/3.99 

TABLE  1.  REGISTRAR  DATA  ON  GOR  AND  GST  CLASSES 
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A  quick  review  of  student  performance  for  the  past  three  graduating  classes 
shows  that  the  success  rate  is  extremely  high.  Only  8  of  127  students  failed  to 
complete  the  program  within  24  months  of  entry.  Two  of  the  eight  who  did  not 
finish  were  transferred  to  the  School  of  Systems  and  Logistics  where  they 
completed  a  different  Master's  degree.  Those  students  who  entered  with  less  than 
3.00  undergraduate  GPAs  performed  almost  as  well  as  those  who  met  the  desired 
standard.  These  data  are  in  Table  2. 


Undergrad  GPA 
less  than  3.00 

Undergrad  GPA 
greater  than  3.00 

Nr  AFIT  GPA 

Nr 

AFIT  GPA 

GOR 

1 

40 

3.58 

GST 

28  3.52 

39 

3.63 

TABLE  2.  COMPARISON  OF  GPAs  (Classes  ot  1984-1986) 

2.  Student  Attitude.  Clearly,  the  GOR  and  GST  students  are  committed  to 
their  studies,  professional  in  their  approach,  and  extremely  hard  working.  Their 
competlveness  and  camaraderie  undoubtedly  enhance  the  learning  environment.  In 
fact,  students  commented  that  they  learn  as  much  from  their  classmates  as  they  do 
in  class.  They  are  supportive  and  display  class  unity.  Their  morale  is  high, 
although  they  have  typical  student  gripes.  Individuals  complained  about  courses 
with  too  much  theory,  others  with  too  much  application,  instructors  who  teach 
"cook  book"  style,  and  Instructors  who  do  not  have  enough  structure  in  their 
classes — evidence  of  differences  in  learning  styles  among  students.  As  a  group, 
however,  they  had  little  tolerance  for  and  were  very  critical  of  a  few  poor 
Instructors,  criticizing  organization,  communications,  pacing,  and  appearance. 

The  committee  generally  views  these  as  typical  student  grips,  which  are  normal 
and  reflective  of  a  healthy,  challenging  program.  The  students  displayed  a 
healthy  thirst  for  knowledge,  but  some  stated  that  they  would  rather  take  more 


courses 


than  research  and  write  a  thesis.  Graduates,  on  the  other  hand,  unanimously 
Indicated  that  their  thesis  work  did  more  than  anything  else  to  tie  the  entire 
program  together  and  to  prepare  them  for  the  complex  problems  they  would 
encounter  In  their  subsequent  duty  tours.  They  wanted  to  specialize  In 
particular  subjects,  Including  network  theory,  stochastic  processes,  artlfical 
Intelligence,  and  response  surface  methodology — clear  evidence  of  the  subject 
stimulation  they  received  in  just  one  year  of  studies.  To  a  person,  they  are 
proud  of  their  program. 

3.  Success  of  Graduates.  The  committee  interviewed  graduates  who  were 
stationed  at  the  Air  Force  Academy  and  Wrlght-Patterson  AFB.  We  also  talked 
with  their  supervisors.  All  graduates  are  succeeding,  although  not  all  are 
employing  the  skills  they  developed  in  the  GOR  or  GST  programs.  For  example, 
some  are  In  units  that  contract  out  studies  and  analysis  problems  the  graduates 
are  eager  to  do  themselves.  All  graduates  said  they  are  very  confident  of  their 
skills,  and  this  was  especially  fostered  by  the  thesis  requirement.  Their 
employers  "could  not  be  more  pleased”  with  their  performance.  They  found 
GOR/GST  graduates  to  be  technically  sound,  adaptive  and  Insightful.  Furthermore 
they  display  an  ability  to  synthesize  and  extend  their  knowledge  to  new  areas  of 
application.  Some  commented  on  AF1T  graduate  strengths  In  computer  skills, 
synthesis,  and  design.  Many  mentioned  a  healthy  competitive  spirit  that  carried 
over  from  their  AFIT  studies  and  many  stated  they  preferred  to  employ  AFIT 
graduates  over  graduates  of  civilian  programs! 


4.  Recommendations .  The  students  are  often  burdened  with  finding  good 
assignments  for  themselves  before  graduation.  This  seems  paradoxical,  because 
these  students  should  be  heavily  recruited  by  agencies  needing  officers  with  a 
state-of-the-art  education  in  operations  research.  We  recommend  that  graduates 
of  these  programs  receive  special  attention  when  it  comes  to  assignments. 

Hiring  agencies  should  be  competing  for  them  much  like  the  situation  in  the 
civilian  community.  This  could  be  done  via  a  "job  fair"  or  a  "career  day"  held 
prior  to  assignment  selection.  As  mentioned  earlier,  the  installation  of  a 
"force  manager"  for  the  268X  and  related  career  fields  at  the  Air  Staff  level 
could  Insure  that  more  attention  is  placed  on  the  proper  use  of  officers. 
Creating  a  Senior  268X  Advisory  Board  consisting  of  the  Coraraanders/Directors  of 
the  major  using  agencies  would  be  comparable  to  the  FA49  Advisory  Board  in  the 
Army.  This  board  should  meet  twice  a  year  and  have  strong  control  over  the  use 
of  268X  officers,  and  hopefully  Influence  the  personnel  system  in  a  very 
positive  way,  especially  through  the  Scientific  Analyst  Career  Monitor  Office 
already  established  at  USAFMPC.  This  board  would  also  relieve  some  of  the 
recruiting  burden  experienced  by  the  department  head  in  hiring  new  instructors. 


D.  Administration  and  Institutional  Commitment 

Cooperation  and  good  communications  abound  among  the  administration  elements 
and  the  faculty  at  the  School  of  Engineering.  Unfortunately,  this  is  threatened 
by  the  lack  of  continuity  brought  about  by  the  rotating  vacancies  in  military 
faculty  and  department  head  positions.  Furthermore,  external  pressure  brought 
about  by  the  personnel  system,  especially  as  mentioned  in  the  faculty  and 
student  body  sections  earlier,  is  causing  problems.  The  financial  support 
provided  to  the  school  seems  adequate.  TDY  budgets  appear  sufficient  for 
professional  development  efforts.  The  library  appears  well  stocked.  The  rapid 
growth  in  computer  and  software  availability  over  the  past  few  years  is  probably 
the  most  visible  sign  of  institutional  support.  The  only  recommendation  we  have 
is  the  call  again  for  high-level  institutional  support  with  the  installation  of 
an  "advocate"  or  "force  manager"  who  could  assist  in  recruiting  top  notch 
faculty  and  overseeing  assignments  for  the  graduates. 


SUMMARY 


The  committee  is  pleased  to  give  the  impressive  GOR  and  GST  programs  an 
excellent  rating.  The  Department  of  Operational  Sciences  has  a  competent 
faculty  and  an  energetic  student  body.  The  core  program,  along  with  the 
elective  sequences  and  the  thesis  requirement,  provides  the  graduates  an 
exceptional  background  compared  to  their  civilian  counterparts.  The  Department 
of  Operational  Sciences  and  its  programs  can  be  made  even  stronger  by  addressing 
the  weaknesses  and  considering  the  recommendations  contained  in  this  report. 

But  these  weaknesses  and  recommendations  do  not  change  the  fact  that  the 
Department  of  Operational  Sciences  is  meeting  its  mission  exceptionally  well. 
This  committee  can  think  of  no  higher  endorsement  than  the  one  given  by  the 
supervisors  of  GOR/GST  graduates:  they  would  be  happy  to  have  as  many  GOR/GST 
graduates  as  they  can  get. 


